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(54) OBJECT DETECTOR, OBSTACLE DETECTOR IN VEHICLE AND VEHICLE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To detect an object by an image 
processing and to further decide kinds of combination of a still object 
and a mobile object by an easy processing. 

SOLUTION: An obstacle is detected by the image processing to 
compare successively photographed image data with a reference 
suj image (background image). First, the number of the detected objects 
is judged whether the number of the detected objects in the previous 
time equals to that of this time (Nt~1=Nt) or not (S10) between two 
pieces of the image data in the previous time and this time and when 
Nt-1 ^Nt, the mobile object is decided to exist. When Nt-1=Nt, the 
number of coincidence of center coordinates is calculated (S60) by 
comparing the center coordinates of the detected objects between 
two pieces of the image data, the number of coincidence equals to 
the detected number (Ne=Nt) or not (S70), when Ne=Nt, only the still 
object is decided to exist (S80) and when Ne^Nt, the mobile object is decided to exist. When the mobile 
object exists, the number of coincidence is further judged whether it is zero (Ne=0) or not (S30, S90), when 
Ne=0, only the mobile object is decided to exist and when the Ne^O, the still object and the mobile object 
are judged to coexist. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1This document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]A sensing device comprising: 

An object detection means which detects an object in said image data by image processing [ reference 

image data / image data / which was photoed one by one / two or more ]. 

A counting means which calculates a detection number of an object in said image data. 

A judging means judged as a movable matter object being included in a detection object when an objective 

detection number is compared between image data of two sheets of last time which differs in photographing 

time, and this time and detection numbers differ. 

[Claim 2]An object sensing device comprising: 

An object detection means which detects an object in said image data by image processing [ reference 

image data / image data / which was photoed one by one / two or more ]. 

A counting means which calculates a detection number of an object in said image data. 

A position calculation means which asks for position data of an object in said image data. 

Compare an objective detection number between image data of two sheets of last time which differs in 

photographing time, and this time, and when a detection number is the same, It asks for the number of 

combination whose position of a detection object corresponds between these 2 sheet image data as the 

number of coincidence using position data called for by said position calculation means, A judging means 

which is judged as a detection object being only a static object when this number of coincidence is the same 

as a detection number, and is judged as a movable matter object being included in a detection object when 

this number of coincidence differs from a detection number. 

[Claim 3]It has a position calculation means which asks for position data of an object in said image data of 
two sheets. When judged with a movable matter object being included in a detection object by said judging 
means, It asks for the number of combination whose position of a detection object corresponds between 
these 2 sheet image data as the number of coincidence using position data called for by said position 
calculation means, The object sensing device according to claim 1 or 2 provided with the 2nd judging means 
that is judged as a detection object being only a movable matter object when this number of coincidence is 
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zero, and is judged as a static object and a movable matter object being intermingled in a detection object 
when this number of coincidence is not zero. 

[Claim 4]An obstacle sensing device in vehicles which are what detects an obstacle of vehicles by image 
processing which compares image data photoed by photographing device which was provided with the 
object sensing device according to any one of claims 1 to 3, and by which said object detection means was 
provided in vehicles with reference image data photoed beforehand. 

[Claim 5]Said photographing device photos vehicles back, and use said photographing device for a specified 
period which can be regarded as said reference image data not having an obstacle in vehicles back in a 
vehicle interdiction, and it is acquired at it, During a vehicle interdiction after a shift operation part was 
switched to a retreated location, said image data is photoed one by one by said photographing device, is 
acquired, and said object sensing device, or [ that an obstacle of vehicles back detected by said object 
detection means includes a movable matter object ] — an obstacle sensing device in the vehicles according 
to claim 4 which distinguish at least whether it is only a static object. 
[Claim 6]Vehicles provided with the obstacle sensing device according to claim 4 or 5. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention detects an object by image processing, and relates to the obstacle 
sensing device and vehicles in the object sensing device and vehicles which recognize even the kind of 
combination of a static object and a movable matter object. 
[0002] 

[Description of the Prior Art]Conventionally, the scene of a vehicle circumference is photoed using the 
camera attached to vehicles, and the object sensing device which detects the obstacle of a vehicle 
circumference by image processing using the photoed image data is known. 

[0003]In JP,5-1 12224A approach of front vehicles is detected from the picture photoed with the camera, 
and the collision avoidance system which operates a brake is indicated. The area change in the picture of 
front vehicles and centroid position change are calculated every moment, and approach of front vehicles is 
judged. The mobile recognition device which detects the movable matter object close to vehicles using 
histogram data during vehicle running is indicated by JPJ-28051 7,A. Each of these detects the object close 
to vehicles during vehicle running. 

[0004]When retreating a car conventionally, the back monitor which projects on the screen of a monitor of a 
driver's seat the image of the vehicles back photoed with the camera is known. It depended for the cognition 
of the situation of vehicles back, and judgment of a safety check on the driver who looked at the image of 
the monitor. That is, the image processing function which detects an obstacle was not provided with the 
back monitor. 
[0005] 

[Problem(s) to be Solved by the Invention]By the way, in order to support the safety check before starting 
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vehicles, for example, there is a case where he would like to detect the obstacle on a course. Even if it could 
detect the movable matter object which approaches vehicles depending on the image processing method of 
the above-mentioned conventional technology, the static object on a course was not able to be detected. In 
order to detect both the static objects and movable matter objects on a course, it is necessary to acquire 
beforehand a background image in case there is no object as a reference image, and image processing, such 
as a finite difference method [ reference image / picture / which was photoed with the camera ], needs to 
detect an object. 

[0006]If the obstacle on a course is a static object, the course which avoids it can also be taken, but when 
it moves also with thinking that the course which avoids it as it is a movable matter object was taken, there 
is a possibility of contacting the object. Therefore, cautions are required especially when an obstacle is a 
movable matter object. Only by seeing the image of a back monitor conventionally, it was dramatically 
difficult to distinguish clearly whether an object is a static object and whether it is a movable matter object. 
Therefore, it is convenient if the purport of the existence of an obstacle can be reported as information 
which detected even the kind of combination of a static object or a movable matter object including the 
obstacle, and included the kind of combination of a static object and a movable matter object in the driver. 
However, in the former, there was no object sensing device which can recognize even the kind of 
combination of a static object and a movable matter object by easy processing about the object detected by 
image processing. 

[0007]It is made in order that this invention may solve an aforementioned problem, and the 1st purpose 
detects an object by image processing, and there is in providing the obstacle sensing device and vehicles in 
the object sensing device which can moreover judge even the kind of combination of a static object and a 
movable matter object by easy processing, and vehicles. The 2nd purpose has a detection object in making 
possible what it divides into all the combination of a static object, a movable matter object, and mixture of 
both, and is judged. 
[0008] 

[Means for Solving the Problemjln order to attain the 1st purpose of the above in the invention according to 
claim 1. An object detection means which detects an object in said image data by image processing 
[ reference image data / image data / which was photoed one by one / two or more ], An objective 
detection number is compared between image data of two sheets of a counting means which calculates a 
detection number of an object in said image data, last time which differs in photographing time, and this time, 
and when detection numbers differ, it has a judging means judged as a movable matter object being included 
in a detection object. 

[0009]An object detection means which detects an object in said image data in the invention according to 
claim 2 by image processing [ reference image data / image data / which was photoed one by one / two or 
more ], A counting means which calculates a detection number of an object in said image data, and a 
position calculation means which asks for position data of an object in said image data, Compare an 
objective detection number between image data of two sheets of last time which differs in photographing 
time, and this time, and when a detection number is the same, It asks for the number of combination whose 
position of a detection object corresponds between these 2 sheet image data as the number of coincidence 
using position data called for by said position calculation means, When this number of coincidence was the 
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same as a detection number, it judged with a detection object being only a static object, and when this 
number of coincidence differs from a detection number, it has a judging means judged as a movable matter 
object being included in a detection object. 

[0010]In order to attain the 2nd purpose of the above in the invention according to claim 3. In the invention 
according to claim 1 or 2, it has a position calculation means which asks for position data of an object in said 
image data of two sheets, When judged with a movable matter object being included in a detection object by 
said judging means, It asks for the number of combination whose position of a detection object corresponds 
between these 2 sheet image data as the number of coincidence using position data called for by said 
position calculation means, When this number of coincidence was zero, it judged with a detection object 
being only a movable matter object, and a detection object is equipped with the 2nd judging means judged as 
a static object and a movable matter object being intermingled when this number of coincidence is not zero. 
[001 1]In the invention according to claim 4, to an obstacle sensing device. Having the object sensing device 
according to any one of claims 1 to 3, said object detection means detects an obstacle of vehicles by image 
processing which compares image data photoed by photographing device provided in vehicles with reference 
image data photoed beforehand. 

[0012]In the invention according to claim 5, it is that in which said photographing device photos vehicles 
back in the invention according to claim 4, Use said photographing device for a specified period which can be 
regarded as said reference image data not having an obstacle in vehicles back in a vehicle interdiction, and 
it is acquired at it, During a vehicle interdiction after a shift operation part was switched to a retreated 
location, said image data is photoed one by one by said photographing device, is acquired, and said object 
sensing device, or [ that an obstacle of vehicles back detected by said object detection means includes a 
movable matter object ] — let it be the gist to distinguish at least whether it is only a static object. 
[0013]Vehicles are equipped with the obstacle sensing device according to claim 4 or 5 in the invention 
according to claim 6. 

(OPERATION) According to the invention according to claim 1, an object in image data is detected by object 
detection means by image processing which compares with reference image data image data photoed one 
by one. A detection number of an object in image data is called for by a measurement means. It judges with 
a judging means including a movable matter object, when a detection number of a detection object is 
compared between image data of two sheets of last time which differs in photographing time, and this time 
and detection numbers differ. 

[0014]According to the invention according to claim 2, an object in image data is detected by object 
detection means by image processing which compares with reference image data image data photoed one 
by one. A detection number of an object in image data is called for by a measurement means. Position data 
of an object in image data is computed by position calculation means. A judging means compares a detection 
number of a detection object between image data of two sheets of last time which differs in photographing 
time, and this time, and if a detection number is the same, Position data of a detection object is compared 
between image data of two sheets, when the same, the number, i.e., number of coincidence, of combination 
whose position of a detection object corresponds, as a detection number, it judges with a detection object 
being only a static object, and when the number of coincidence differs from a detection number, it judges 
with a movable matter object being included in a detection object. 
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[0015]According to the invention according to claim 3, position data of an object in image data of two sheets 
is computed by position calculation means. When the 2nd judging means is judged as a movable matter 
object being included in a detection object by judging means, Position data of a detection object is 
compared between image data of two sheets, when the number of combination whose position of a 
detection object corresponds, i.e., the number of coincidence, is zero, a detection object judges only with a 
movable matter object, and when the number of coincidence is not zero, it judges with a static object and a 
movable matter object being intermingled in a detection object. 

[0016]According to the invention according to claim 4, an object sensing device detects an obstacle of 
vehicles by an object detection means by image processing which compares image data photoed by 
photographing device provided in vehicles with reference image data photoed beforehand. And it is judged to 
which combination of a static object and a movable matter object the obstacle corresponds. 
[0017]According to the invention according to claim 5, an obstacle sensing device, Reference image data in 
which a photographing device photoed and obtained vehicles back in a vehicle interdiction at a specified 
period which can be regarded as there being no obstacle in vehicles back, After switching a shift operation 
part to a retreated location in the vehicle interdiction, image processing to which a photographing device 
carries out the successive approximation of the image data which obtained vehicles back by photoing it one 
by one detects an obstacle of vehicles back, or [ and / that an obstacle of detected vehicles back includes 
a movable matter object ] — it is distinguished at least whether it is only a static object. 
[0018]Since vehicles are provided with the obstacle sensing device according to claim 4 or 5 according to 
the invention according to claim 6, the same operation as the invention according to claim 4 or 5 is obtained. 
[0019] 

[Embodiment of the Invention]Hereafter, one embodiment which materialized this invention is described 
based on drawing 1 - drawing 7 . As shown in drawing 7 , the camera 2 as a photographing device is attached 
to the upper rear part position of the car 1 as vehicles. The camera 2 contains a CCD element, and when 
the car 1 retreats, it photos the back field shown in the figure which covers at least the range which needs 
a back safety check with the broken chain line. The car-navigation system is carried in the car 1, and LCD 
monitor 3 for car navigation is formed for example, on the instrument panel near the drivers seat. 
[0020]According to this embodiment, the image (photography video) of the back field of the car 1 photoed 
with the camera 2 is displayed using LCD monitor 3 for car navigation at the time of vehicles retreat. The 
camera 2 is attached on the shooting posture square as which the rear bumper 4 is displayed in part at the 
lower end of the screen of LCD monitor 3 as shown in drawing 6 (see drawing 7 ). The retreat support device 
5 (shown in drawing 4 ) is carried in the car 1. If image processing using the picture photoed with the camera 
2 detects whether an obstacle exists in vehicles back and the retreat support device 5 has an obstacle in it 
before a retreat start, it will report that and will support a driver s back safety check. The camera 2 and LCD 
monitor 3 are making some component parts of the retreat support device 5. 

[0021] Drawing 4 is a block diagram showing the electric constitution of the retreat support device 5. The 
retreat support device 5 The camera 2, ECU(electronic control) 6 for pictures, ECU7 for car navigation, an 
LCD monitor. (A monitor is only called hereafter) It has 3, the shift position switch 8, the door switch 9, the 
speed sensor 1 0, the steering piece angle sensor 1 1 , the voice processing circuit 1 2, and the loudspeaker 1 3 
ECU7 for car navigation and the monitor 3 which are component parts of a car-navigation system are 
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diverted to the retreat support device 5. The monitor only for retreat support can also be prepared. 
[0022]The camera 2 is electrically connected to ECU6 for pictures, and the image data picturized by the 
CCD element is transmitted to ECU6 for pictures. As for the camera 2, turning on and off of photography is 
controlled by ECU6 for pictures. 

[0023]While the shift position switch 8, the door switch 9, and the speed sensor 10 are connected via the 
input port 14, the steering piece angle sensor 1 1 is connected to ECU6 for pictures via A/D converter 15. 
ECU7 for car navigation and the voice processing circuit 12 are connected to ECU6 for pictures. The 
monitor 3 is connected to ECU7 for car navigation, and the loudspeaker 13 is connected to the voice 
processing circuit 12. 

[0024]The shift position switch 8 detects the actuated valve position of the shift lever 16 (shown in drawing 
7), and ECU6 for pictures recognizes shift position SP by the output signal SPS. The car 1 of this 
embodiment is an automatic car, and there are six kinds of shift position SP "P, R, N, D, 1st speed, and 2nd 
speed." 

[0025]The door switch 9 detects opening and closing of the door 17 (shown in drawing 5 ) of a driver's seat, 
and outputs door-signals DS turned on and off according to opening and closing of the door 17. When 
turning on and off of door-signals DS changes, opening and closing of the door 17 are detected. 
[0026]The speed sensor 10 detects the vehicle speed and outputs the detecting signal vS according to the 
vehicle speed v. The steering piece angle sensor 1 1 detects the piece angle of the steering wheel 18 (shown 
in drawing 7 ). The steering piece angle sensor 1 1 detects the rotation of the steering shaft (not shown) 
which supports the steering wheel (handle) 19 (shown in drawing 7 ), for example, and outputs 
detecting-signal thetaS according to the piece angle theta of the steering wheel 1 8. 
[0027]ECU7 for car navigation is because it is built in a car-navigation system and control of a 
car-navigation system is managed. ECU7 for car navigation carries out image display control of the monitor 
3 carried in the car-navigation system. According to this embodiment, ECU7 for car navigation also 
performs display control of image data by which an external input is carried out from ECU6 for pictures. 
[0028]ECU6 for pictures is provided with the central processing unit (henceforth CPU) 20, the image 
memory 21, and the memory (for example, ROM and RAM) 22. CPU20 operates based on the program data 
memorized by the memory 22. An object detection means, a counting means, a judging means, a position 
calculation means, and the 2nd judging means are constituted by ECU6 (CPU20) for pictures. 
[0029]The program data for performing for example, retreat support processing in the memory 22 is 
memorized. From the time of a driver considering that there is no obstacle in back, opening and closing the 
door 17, and getting in a driver's seat with retreat support here. It is a system which supports a back safety 
check until it begins retreat because a sound and a picture report that for image processing to detect the 
change of a situation of vehicles back, and to detect an obstacle on a retreat course to a driver until the car 
1 begins to move back. 

[0030]According to this embodiment, the detected obstacle distinguishes a certain thing and the thing 
which entered in the retreat course on a retreat course, and reports to a driver. Therefore, detection of an 
obstacle will perform obstacle recognition processing which identifies to which an obstacle corresponds 
among three kinds of combination of whether a static object, a movable matter object, a static object, and a 
movable matter object are intermingled. CPU20 is provided with the buffer for saving temporarily the data 
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used for the obstacle recognition processing which identifies to which a detection object corresponds 
among said three kinds of combination. The data saved at the buffer is used also for the rectangle 
superimposed display processing mentioned later. 

[0031]The program data of the obstacle recognition processing shown in drawin g 1 with a flow chart is 
memorized by the memory 22. Two kinds of voice data are memorized by the memory 22. The program data 
of the rectangle superimposed display processing for furthermore carrying out the superimposed display 
(superimposed title) of the rectangle surrounding the obstacle detected in the image projected on Screen 3a 
of the monitor 3 by an animation when shift position SP is "R" to the memory 22 is memorized. At rectangle 
superimposed display processing, an obstacle displays the rectangle surrounding an obstacle by two kinds of 
display modes with a static object or a movable matter object. In detail, a rectangle blinks about a static 
object, and it is displayed so that a rectangle may follow in footsteps of movement [ an object ] and may 
move about a movable matter object. 

[0032] Drawing 5 is an explanatory view showing the flow of the obstacle detection in retreat support 
processing. The program of retreat support processing will be executed by CPU20 if it detects that the car 
1 stopped (v= 0). First, the camera 2 is operated and reference image Imageof one sheetO is acquired 
beforehand at the specified period which can be regarded as there being no obstacle in vehicles back. As a 
specified period which acquires reference image ImageO, the time of operating the shift lever 16 in parking 
(parking position) "P" or a neutral (center valve position) "N" is adopted. Whenever the door 17 of a driver's 
seat is opened and closed, reference image ImageO is updated. That is, the time of opening and closing of 
the door 17 of a drivers seat is adopted as vehicles back immediately after stopping and parking as a stage 
(time to) which can be regarded as there being no obstacle. 

[0033]Since the picture of the vehicles back at that time is equivalent to the place vehicles have run now 
immediately after stopping and parking, it can be considered that there is no obstacle in the vehicles back 
immediately after stopping and parking. Since a driver usually operates the shift lever 16 to "P" or "N" 
immediately after stopping and parking, he is made to acquire a back picture when the shift lever 16 is 
operated by "P" or "N" as reference image ImageO after a stop (vehicle speed v= 0). 
[0034]It is at the driver's getting-on-and-off time at the time of opening and closing of the door 17 of a 
driver's seat. Usually, since a driver checks the back of vehicles once when it gets off, business is finished 
and it has returned to vehicles, he can consider that there is no obstacle in vehicles back. After parking and 
stopping, reference image ImageO is updated because there is a time of time having passed by the time it 
begins retreat. At this time, light sources, such as a difference between daytime and night, have a big 
change, and when a back picture when the shift lever 16 is operated by "P" or "N" is set to reference image 
ImageO, there is a possibility that erroneous detection may arise. Therefore, he is trying to update a back 
picture when the door 17 is opened and closed as reference image ImageO. 

[0035]And if the shift lever 16 is put into "R" (time t1), a photograph will be taken one by one to every 
predetermined time deltat (for example, 33-66 milliseconds), and the comparison picture Imagen (n= 1, 2 and 
3, — ) will be acquired. Whenever it acquires the one comparison picture Imagen, image processing [ Imagen 
/ picture / reference image ImageO and / comparison ] is performed, and an obstacle is detected. The 
obstacle furthermore detected identifies to which combination it corresponds among a "static object", a 
"movable matter object", and a "static object + movable matter object." Only between until vehicles begin 
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to move is performed, and retreat support processing will suspend processing, if the vehicle speed v is no 
longer zero (v!=0). Processing is suspended also when the shift lever 16 is changed into positions other than 
"R." However, even if it suspends retreat support processing, while being put into shift position SP by "R", 
photography with the camera 2 is continued and a back image projects on Screen 3a of the monitor 3. 
[0036]Next, difference processing is explained in detail. According to this embodiment, image processing 
according picture ImageO and Imagen of two sheets to a background finite difference method is carried out. 
For example, the absolute value (|lmagen-ImageO|) of the difference of Imagen and ImageO is taken, and 
expansion processing is performed to the difference image. Even if one object is divided into plurality by this 
in the part (for example, dark space by a shadow) where luminosity differs, it combines with the one original 
object. Next, the threshold process using a predetermined threshold is performed and a binary picture is 
obtained. In binary picture data, a portion which is different by ImageO and Imagen is expressed as a white 
region (data "1 "). 

[0037]Next, the processing which specifies an obstacle from a detection object is explained. He is trying to 
detect only the object of anticipation retreat course PA in case vehicles retreat within the limits as an 
obstacle, as shown in drawing 6 . Based on the piece angle theta of the steering wheel 18 obtained from 
signal value thetaS from the steering piece angle sensor 11, anticipation retreat course PA on a binary 
picture is calculated. A formula is used for calculation of anticipation retreat course PA for example, the 
memory 22 memorized beforehand. In consideration of the distance on a binary picture, the equation of two 
or more curves, such as an ellipse and a parabolic curve, etc. are used for a formula. Drawing 6 is what 
showed the binary picture Image typically, and anticipation retreat course PA is calculated as a field across 
which it faced with two lines (a broken chain line shows in the figure) decided from a formula. As shown in 
drawing 6 (a), in the piece angle theta at the time of rectilinear propagation (=0), anticipation retreat course 
PA serves as a field where it extends straight as a long distance so that width may become narrow. As 
shown in drawing 6 (b), in the piece angle theta (theta> 0 or theta< 0) when the wheel 19 is turned, 
anticipation retreat course PA serves as a field on which width draws the curve which becomes narrow like 
a long distance. The width of anticipation retreat course PA is almost equal to breadth of a car. By drawing 
8ja), it is black and the detection object shown in a white region is shown. 

[0038]It is judged whether a detection object predicts the size and it can become an obstacle. Actual size 
detects only the thing exceeding prescribed size as an obstacle. As shown in drawing 6, the inside of 
anticipation retreat course PA in the binary picture Image is first divided into the block B of plurality (for 
example, three) which responded to the far and near difference, Size comparison with the area value of a 
detection object (white region) and threshold which exist also partly in the block B at each the block B of 
every is carried out using the threshold set to each the block B of every. When the area of a white region 
exceeds a threshold, it judges with the object of the white region being the object of the size which can 
serve as an obstacle, i.e., an obstacle. About the detection object (white region) judged to be an obstacle, 
the detection number, the center coordinates of each white region, and the rectangular coordinates, width 
and height surrounding each white region of the starting point (top left point) are found, and the data is 
memorized to a buffer. To a buffer, the data for the five past is memorizable including this time. Field OA is 

not made into the object of obstacle detection outside besides the range of anticipation retreat course PA 
shown in d raw ing 6, and the object which exists in outside field OA is not detected as an obstacle. 
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[0039]Next, it explains, using drawing 2 and drawing 3 based on the flow chart of drawing 1 about obstacle 
recognition processing. When an obstacle is detected, it progresses to obstacle recognition processing. In 
obstacle recognition processing, the data memorized to the buffer by each obstacle detection processing of 
last time (time t-1) and this time (time t) is used, and it is identified to which an obstacle corresponds 
among three kinds of combination, a "static object", a "movable matter object", and a "static object + 
movable matter object." 

[0040]The data of the detection number of an obstacle and the data of the center coordinates of each 
obstacle in last time and this time are first read from a buffer. Here, the detection number of the obstacle in 
the last time t-1 and this time t is set to Nt-1 and Nt, respectively. The center coordinates of each obstacle 
in each time t-1 and t are set to Ck, t-1 (however, k= 1, 2, — , Nt-1), Ck, and t (however, k= 1, 2, — , Nt), 
respectively. Here, the number which specifies an obstacle [ in / in the front subscript k / each frame ], and 
a next subscript show time. The following processings are performed one by one using these two kinds of 
data. 

[0041]First, at Step 10, a detection number is compared and it is judged whether it is Nt-1=Nt. If Nt-1 !=Nt 
becomes, it will judge with "those with a movable matter object." It is because the change in the detection 
number in the time t and the time t-1 is certainly brought about with a movable matter object when 
detection numbers differ at the time t-1 and the time t (Nt-1 !=Nt) as shown in drawing 2 . The processing 
concerned is equivalent to the judging means which judges based on comparison of a detection number. 
[0042]Since processing of Step 20 - Step 50 was Nt-1 !=Nt, when it judges with "those with a movable 
matter object", it is processing for distinguishing a "movable matter object" and a "static object + movable 
matter object", and is equivalent to the 2nd judging means. Step 20 compares the center coordinates of the 
obstacle in each time t-1 and t. Center coordinates are compared about all the combination of an obstacle 
between the image data of the time t and the time t-1. For example, if the center coordinates Ck of the time 
t-1 , t-1 and the center coordinates Ck of the time t, and combination with t are set with (Ck, t-1 , Ck, t), (C1 , 
t-1, C1, t), (C1, t-1, C2, t), — , (C1, t-1, CNt, t), It investigates about all the combination like (C2, t-1, C1, t), 
(C2, t-1, C2, t), — (C2, t-1, CNt, t), — , (CNt-1, t-1, C1, t), (CNt-1, t-1, C2, t), — , (CNt-1, t-1. CN, t) It 
supposes that it is obtained whether comparison of center coordinates is in agreement, and asks for the 
number Ne of the combination which investigates all the combination and whose center coordinates 
correspond (henceforth the number of coincidence). When there is no detection object at one of the time t 
and the time t-1 at this time (in the case only of a movable matter object), suppose that center coordinates 
are compared as (0, 0). The position and size of the obstacle (white region) detected are giving a certain 
amount of tolerance level to the judgment of coincidence of center coordinates in consideration of the error 
which swings by the difference in how depending on which light hits, etc. 

[0043]In Step 30, it is judged whether it is number Neof coincidence =0. Since the center coordinates of 
each object will change at the time t and the time t-1 if objective [ all ] are movable matter objects as 
shown in drawing 2 (a) and (b), the number Ne of coincidence becomes zero (Ne=0). Since the center 
coordinates of a static object do not change but are in agreement at the time t-1 and the time t when a 
static object is intermingled as shown in drawing 2 (c), the number Ne of coincidence is no longer zero 
(Ne!=0). Therefore, if Ne=0 is materialized, only a "movable matter object" will be judged at Step 40, and if it 
is Ne!=0, it will judge with a "static object + movable matter object" at Step 50. 
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[0044]On the other hand, in Step 10, when it is Nt-1=Nt, it cannot be identified only now to which it 
corresponds among three kinds of combination. In Nt~1=Nt, processing of Steps 60 and 70 is processing for 
distinguishing a "static object" and "those with a movable matter object" first. This processing is equivalent 
to the judging means which judges when a detection number is the same. 

[0045]Step 60 compares the center coordinates of the obstacle between each time t-1 and t. The contents 
of processing are the same as Step 20, and this investigates them about all the combination. It supposes 
that it is obtained whether comparison of center coordinates is in agreement, and asks for the number Ne of 
the combination which investigates all the combination and whose center coordinates correspond 
(henceforth the number of coincidence). 

[0046]In Step 70, it is judged whether it is that the number Ne of coincidence is equal (Ne=Nt) to the 
detection number Nt. Since the center coordinates of each object will not change at the time t and the time 
t-1 if objective [ all ] are static objects as shown in drawing 3 (a), the number Ne of coincidence becomes 
equal to the detection number Nt (=Nt~1). Since the center coordinates of a movable matter object change 
at the time t-1 and the time t when a movable matter object exists as shown in drawing 3 (b) and (c), the 
number Ne of coincidence certainly becomes less than the detection number Nt. Therefore, if Ne=Nt (the 
example of drawing 3 (a) Ne=Nt=2) is materialized, only a "static object" will be judged at Step 80. On the 
other hand, if Ne=Nt is not materialized (in the case of drawing 3 (b) and (c)), it progresses to Step 90. 
[0047]That is, a case "with a movable matter object" will progress to Step 90 in the time of Nt-1=Nt. By 
Nt-1=Nt, in the case of "with a movable matter object", processing of Step 90 - Step 1 10 is processing for 
distinguishing a "movable matter object" and a "static object + movable matter object", and is equivalent to 
the 2nd judging means. In Step 90, it is judged whether it is number Neof coincidence =0. Since the center 
coordinates of each object will change at the time t and the time t-1 if objective [ all ] are movable matter 
objects as shown in drawing 3 (b), the number Ne of coincidence becomes zero (Ne=0). Since the center 
coordinates of a static object do not change but are in agreement at the time t-1 and the time t when a 
static object is intermingled as shown in drawing 3 (c), the number Ne of coincidence is no longer zero 
(Ne!=0). Therefore, if Ne=0 is materialized, only a "movable matter object" will be judged at Step 100, and if 
it is Ne!=0, it will judge with a "static object + movable matter object" at Step 1 10. 

[0048]Next, an operation of the retreat support device 5 is explained. If vehicles stop and it is put into the 
shift lever 16 by "N" and "P", reference image Imageof one sheetO will be photoed with the camera 2. Then, 
whenever the door 17 of a driver's seat is opened and closed, reference image ImageO is updated. The 
picture of the nearest stage that can be regarded as there being no obstacle in vehicles back by these 
processings is memorized by the image memory 21 as reference image ImageO. 

[0049]Here, in the case of stopping and parking, by the time it next begins to retreat vehicles, a driver may 
get down from vehicles, or it may not get down. When a picture when a driver does not get down from 
vehicles and the shift lever 16 is put into "P" or "N" is set to reference image ImageO and a driver gets 
down from vehicles, a picture when the door 17 of a drivers seat is opened and closed is set to reference 
image ImageO. 

[0050]Next, if the shift lever 16 is put into "R" in order that a driver may retreat vehicles, the image of the 
vehicles back photoed with the camera 2 will project on Screen 3a of the monitor 3. For example, Screen 3a 
of the monitor 3 switches to the image of car navigation screen empty vehicle both back. Obstacle 
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detection processing is performed at the same time it is put into the shift lever 16 by "R." 
[0051]Namely, the comparison picture Imagen (n= 1, 2, — ) is photoed by every predetermined time deltat, 
Image processing which uses a background finite difference method at every time, and compares reference 
image ImageO and the comparison picture Imagen (n= 1 t 2, — ) with it is performed, and obstacle detection is 
performed within the limits of anticipation retreat course PA decided from the piece angle theta of the 
steering wheel 18. If an obstacle is detected in anticipation retreat course PA, obstacle recognition 
processing will be performed. 

[0052]In Step (it is hereafter described as "S") 10, it is judged first whether it is Nt-1=Nt. For example, as 
shown in drawing 2 , in Nt-1 !=Nt, it progresses S20, and as shown in drawing 3 , in Nt-1=Nt, it progresses S60. 
[0053]In Nt-1 !=Nt, S20 compares center coordinates first and it asks for the number Ne of coincidence. In 
S30, it is judged whether it is number Neof coincidence =0, As shown in drawing 2 (a) and (b), in the case of 
|\| e =o, only a "movable matter object" is judged (S40), and as shown in drawing 2 (c), in the case of Ne!=0, it 
is judged with a "static object + movable matter object" (S50). 

[0054]On the other hand, in Nt-1=Nt, S60 compares center coordinates first and it asks for the number Ne 
of coincidence. In S70, it is judged whether it is number Ne=Nt of coincidence. In Ne=Nt, only a "static 
object" is judged as shown in drawing 3 (a). As shown in drawing 3 (b) and (c), in Nel^Nt, it progresses to S90. 
In S90, it is judged whether it is Ne=0. As shown in drawing 3 (b), in the case of Ne=0, only a "movable 
matter object" is judged (S100), and as shown in drawing 3 (b), in the case of Ne!=0, it is judged with a 
"static object + movable matter object" (S1 10). In this way, it is judged any of a "static object", a "movable 
matter object", and a "static object + movable matter object" obstacles are by obstacle recognition 
processing. In the case of a "static object + movable matter object", it restricts, and processing which 
specifies a static object and a movable matter object is performed. That is, a detection object with the 
combination whose center coordinates correspond is judged using the result data at the time of 
center-coordinates comparison to be a static object, and a detection object without the combination whose 
center coordinates correspond is judged to be a movable matter object. 

[0055]"An obstacle is on a course" is warned with a sound from the loudspeaker 13, and on the other hand, 
when it is "with a movable matter object", "An obstacle goes into a course" is warned of the case of only a 
"static object" with a sound from the loudspeaker 13. And the rectangle surrounding an obstacle is 
displayed into the image of Screen 3a, and the rectangle blinks in a static object, and with a movable matter 
object, it moves so that it may follow in footsteps of movement [ an object ]. For this reason, it turns out 
that that obstacle is a static object because the rectangle in the image of Screen 3a blinks, and when a 
rectangle follows in footsteps of an obstacle shows a motion of a movable matter object well. 
[0056]If vehicles begin to retreat (vehicle speed v!=0), the program of retreat support processing will be 
ended, but the image of vehicles back projects on Screen 3a of the monitor 3 succeedingly during retreat of 
vehicles. If vehicles retreat is finished and the shift lever 1 6 is switched to other positions (for example, "P", 
"N", etc.) from "R", Screen 3a of the monitor 3 will be switched to the original picture from the image of 
vehicles back. 

[0057]According to this embodiment, the following effects are acquired as explained in full detail above. 
(1) the detection number of a detection object is compared between the time t-1 and the image data of two 
sheets oft (S10), and a detection number is not equal, since it judges with "those with a movable matter 

12 



JP 2000-182027 

object" at the time (Nt-1 !=Nt), When a movable matter object disappears from a photography screen or it 
newly enters (drawing 2 (a)), the judgment "with a movable matter object" can be simplified. 
[0058](2) As a result of comparing the detection number of a detection object between the time t~1 and the 
image data of two sheets oft (S10), when a detection number is equal (Nt-1=Nt), Furthermore 
center-coordinates comparison is carried out and it asks for the number of coincidence (S60), and when the 
number of coincidence is equal to a detection number (Ne=Nt) and a "static object" and the number of 
coincidence are not equal to a detection number (Ne!=Nt), it can judge with "those with a movable matter 
object." therefore, a detection object is distinguishable to a "static object" and "those with a movable 
matter object" by all the cases by two decision processings of detection number comparison processing 
(Nt-1-Nlt — do formation or not?) (S10) and the number decision processing of center-coordinates 
coincidence (is it Ne=Nt or not?) (S60, 70). 

[0059](3) In" with "movable matter object a "movable matter object" and a "static object + movable 
matter object" are distinguishable by processing (S30, S90) which judges whether it is that the number of 
coincidence of center coordinates is zero (Ne=0). When carrying out the superimposed display of the 
rectangle to the obstacle on the image of Screen 3a of the monitor 3, in order to decide the display mode 
(blink or imitation), it is necessary to specify a static object and a movable matter object. Even in this case, 
since "those with a movable matter object" can be further divided into a "movable matter object" and a 
"static object + movable matter object", Only in the case of a "static object + movable matter object", 
processing which specifies a static object and a movable matter object is performed, the futility of 
processing can be excluded, and the burden of processing of CPU20 can be eased. Since processing which 
specifies a static object and a movable matter object is performed only at the time of necessity, since 
specific processing was carried out at the time only of a "movable matter object", a specific mistake which 
mistakes a movable matter object for a static object can be made, and the display mistake which displays a 
rectangle on a movable matter object by the display mode of a static object can be lessened. 
[0060](4) In order to retreat vehicles, when the shift lever 16 is operated to "R" and it is not getting down 
after a driver makes vehicles park and stop, From the time of putting the shift lever 16 into P or N , 
when a driver gets down from vehicles, Since the door 17 was opened and closed and the change of a 
situation (existence of an obstacle) of vehicles back until vehicles begin to move is reported to a driver 
through a sound and the superimposed display (superimposed title) in the image of the monitor 3, the back 
safety check before a driver retreats the car 1 is supportable. 

[0061](5) What kind of thing is an obstacle? Since two kinds of announced information (the contents of a 
voice, display mode) were prepared according to the difference between a "static object" and "with a 
movable matter object", by information of a sound or a picture, a driver tends to grasp the character and 
trend. 

[0062]An embodiment is not limited above but can also be carried out in the following modes. 
O The method of distinguishing at a "static object" and a "static object + movable matter object" in the 
case of "with a movable matter object" is not limited to said embodiment. [ which is judged to be Nt~1 !=Nt 
in S10 ] For example, Step 15 may be added as shown in drawing 8 . In Step 15, it is judged whether Nt~1=0 
or Nt=0 are materialized. As shown in drawing 2 (a), when there is no detection object in one picture among 
the time t-1 and the time t, only a "movable matter object" can be judged by easy processing in which a 
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detection number is counted, without comparing center coordinates. 

[0063]O According to said embodiment, although three kinds, a "static object", a "movable matter object", 
and a "static object + movable matter object", were identified, it is not limited to three kinds of discernment. 
It is also good to identify two kinds such as a "static object" and "those with a movable matter object." For 
example, the flow chart shown in drawing 9 is employable. That is, when Nt~1=Nt is abortive in S210, it 
judges with "those with a movable matter object" by S220. When it is judged that they are not a "static 
object" and Ne=Nt in S250 when the number Ne of coincidence for which it asked by S230 is judged to be 
Ne=Nt in S240, it judges with "those with a movable matter object" by S260. 

[0064]O The position data for comparing the position of a detection object is not limited to center 
coordinates. It is sufficient if it is coordinates of one point which can pinpoint the position of a detection 
object also except center coordinates. For example, the coordinates of the center of gravity decided from 
the shape of a detection object may be sufficient. If it is coordinates of one point, the comparison 
processing of a position will be easy and will end. As long as it can pinpoint the position of a detection object, 
position data may be coordinate data which specifies a detection object in the range in addition to 
coordinates (one point), for example. Position data other than coordinates can also be used, and it is 
sufficient if it is data which can pinpoint the position of a detection object in short. 
[0065]O The detection object range of an obstacle is not limited in anticipation retreat course PA. For 
example, the whole (picture total range) photographing area can also be made into an objective detection 
object range. A detection object range may be narrowed on the contrary. For example, only prescribed 
distance within the limits is made into an object range from vehicles, and processing speed can be brought 
forward if image processing only of the field is carried out. About a movable matter object, the range of 
constant distance can be set outside as for detection, for example from anticipation retreat course PA, and 
the composition which discovers beforehand [ before going into a course ] the movable matter object which 
tries to enter in a retreat course, and reports it can also be taken. 

[0066]O Announced information (information mode) is not limited to two kinds of distinction, a "static 
object" and "those with a movable matter object." Three kinds of announced information (information 
mode) according to three kinds of combination, a "static object", a "movable matter object", and a "static 
object + movable matter object", are prepared, one according to a decision result of three kinds is chosen, 
and the announced information can report. Of course, if required, a "movable matter object" and two kinds 
of announced information "with a static object" (information mode) can also be prepared. 
[0067]O An informing method can be changed suitably. It is considered only as voice warning or is good only 
also as a superimposed display (superimposed title) to the image of Screen 3a of the monitor 3. Not only a 
sound but furthermore, a buzzer, a chime, etc. may be sufficient as the warning method by a sound, for 
example. Announced information is distinguishable if the mode of a sound is changed. 

[0068]O It is made to perform obstacle detection at night by attaching the function of an infrared camera. 
O A retreat support device is widely applicable to cars, such as a bus and a track. It may apply to industrial 
truck, such as a fork lift truck. It is not limited to an automatic car but can apply also to a car with a manual 
transmission. 

[0069]O It is not limited to applying to the retreat support device of vehicles. It is applicable to the obstacle 
sensing device which detects the obstacle of the front of vehicles, or the side. In this case, before vehicles 
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begin to move, the safety check of the direction by which vehicles begin to run is supportable. 
[0070]O Vehicles are not limited to manned vehicles, such as a car. For example, it may apply to unmanned 
vehicles, such as a unmanned folk lift and an automatic guided vehicle, or [ in this case, / that that obstacle 
includes a movable matter object by detecting the obstacle on a course at the time of start ] — start 
control and information control are carried out only with a static object. For example, only when a movable 
matter object is included in an obstacle, fixed time standby is carried out and an obstacle still exists, the 
purport of obstacle detection is reported to a worker, and only in the case of a static object, an obstacle 
judges it as falling objects, such as a load, and reports the purport of obstacle detection to a worker 
promptly. It applies to the unmanned vehicle of the method it runs while choosing a course freely without 
the course of an owner orbit, and an obstacle is detected and it may be made to choose the course of start 
by image processing at the time of start. For example, only in the case of a static object, an obstacle 
departs with the course which avoids an obstacle, and a start course may be decided after carrying out 
fixed time standby, when it includes a movable matter object. 

[007 1]0 It is not limited to application to the obstacle sensing device of vehicles. For example, it may apply 
to a security camera. A photograph is made to usually take with a camera beforehand by using the 
background image at the time as reference image data in the case of a security camera, and what is 
necessary is just to make it memorize. 

[0072]Technical ideas (invention) other than the claim grasped from said embodiment and each example of 
another are indicated below. 

(1) In claim 2, said judging means also carries out the contents of processing of said judging means in claim 
1. In this case, it is not based on the difference of the object detection number between the image data of 
two sheets, but it can be judged whether a detection object includes a static object and a movable matter 
object. 

[0073](2) When judged with a movable matter object being included in a detection object in claim 1 since a 
detection number changes with said judging means, When it judges whether there is any detection object in 
one side of said image data of two sheets and judges that there is no detection object in one image data, it 
has the 3rd judging means judged as a detection object being only a movable matter object. In this case, 
when are judged with including a movable matter object by a judging means and there is no detection object 
in one side of image data, it can be judged whether it is only a movable matter object by easy processing in 
which a detection number is seen. The 3rd judging means is constituted by ECU6 (CPU20) for pictures. 
[0074](3) In either claim 2 - claim 6, said position data is coordinates which pinpoint an objective position. In 
this case, it ends with easy processing of only comparison of coordinates (point). 

(4) Claim 1 - claim 6, and the above (1) In either of the technical ideas of - (3), said image processing 
compares said image data with said reference image data by difference processing. 

[0075](5) Claim 1 - claim 6, and the above (1) In either of the technical ideas of - (4), said image data is 
obtained by being photoed one by one by the photographing device stationary at least at the time of 
photography. 

[0076](6) In the technical idea of the above (5), said reference image data is the background image photoed 
beforehand, when there is no object by said photographing device. 

(7) In the technical idea of the above (6), it is the background image beforehand photoed by this 
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photographing device at the specified period which can be regarded as said reference image data not having 
an obstacle (object) into the photographing area of said photographing device. According to the technical 
idea of (4) - (7), the respectively same effect as the invention of a quoting agency or a technical idea is 
acquired above. 

[0077](8) Claim 1 - claim 3, and the above (1) In either of the technical ideas of - (7), said object sensing 
device is an object for vehicles, and said reference image data and said image data are photoed during a 
vehicle interdiction by the photographing device provided in vehicles in the circumference so that 
photography was possible, or [ in this case, / including a movable matter object for the object (obstacle) of 
a vehicle circumference during a vehicle interdiction (at the time of vehicle departing) ] — only a static 
object is distinguished at least and can be detected. 

[0078](9) In the technical idea of the above (8), said photographing device takes a photograph during said 
vehicle interdiction, and said object detection means detects the object (obstacle) of a vehicle 
circumference during a vehicle interdiction. In this case, the same effect as the technical idea of a quoting 
agency is acquired. 

[0079](10) Claim 1 - claim 6, and the above (1) In either of the technical ideas of - (9), Said object detection 
means's detection of an object (obstacle) is provided with the informing means which reports that the 
object (obstacle) was detected in the information mode chosen from two or more sorts of information 
modes according to the decision result of said judging means, or [ in this case, / including a movable matter 
object for an obstacle existing ] — only a static object is distinguished and it can report to a driver. 
[0080](1 1) in the technical idea of the above (10), said informing means is provided with the image display 
device which projects the image of said image data by an animation, and carries out the superimposed 
display of the specified display to the object (obstacle) on the image of the screen by a different display 
mode according to the decision result by the aforementioned judging means (each). An image display device 
is constituted by the monitor 3 in said embodiment, or [ in this case, / that an object (obstacle) includes a 
movable matter object by seeing the specified display on the image of a screen ] — only a static object is 
distinguished at least and can be recognized. 
[0081] 

[Effect of the InventionjThat a movable matter object is included in the object detected by image 
processing which compares with reference image data the image data photoed one by one according to the 
invention according to claim 1 to 6 as explained in full detail above, It can judge by easy processing in which 
the detection number of a detection object is compared between the image data of two sheets. 
[0082]According to the invention according to claim 2 to 6, the object detected by image processing which 
compares with reference image data the image data photoed one by one, The position of a detection object 
is compared between the image data which is [ whether a static object and a movable matter object are 
included and ] two sheets when an objective detection number is the same, and it can judge by easy 
processing which judges whether the number of coincidence of a position (the number of combination) is the 
same as a detection number. 

[0083]According to the invention according to claim 3 to 6, when it includes a movable matter object, the 
position of a detection object is compared between the image data of two sheets, and it can be judged 
whether a movable matter object and a static object are intermingled by easy processing which judges 
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whether the number of coincidence of a position (the number of combination) is zero. Therefore, a detection 
object divides into a static object, a movable matter object, and all the combination of whether both are 
intermingled, and it can judge. 

[0084]or [ that a movable matter object is included in the obstacle of vehicles according to the invention 
according to claim 4 to 6 ] — only a static object is distinguished at least and can be detected, or [ that a 
movable matter object is included in the obstacle of vehicles back during the stop which switched the shift 
operation part to the retreated location according to claim 5 and the invention according to claim 6 ] only 
a static object is distinguished at least and can be detected. 



[Translation done.] 
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txtzt><DX'&*), ^com 1 £>l#Jt±. EtfcMSfc ±ot 
M*;&fttBU Lfr&fS#&©SlcJ;oTt£±1%ft£^ 

j«, iltWfcfcttSWWftftttliSif'f&tf »PM%*B0WS C 10 

<ttc$,^ 0 jfi2^ne^t±, ftwi*f**v 

[0 0 0 8] 

Si:, fiis-gH^iJODi'^ 3 fjijlul £4t.'l<D 2 |ft<D|iiii#l-r— £ 20 

[0 0 0 9] Igj|<rM2tc|2*^?g0^T'ii, iS^JgigSft 

S i: . S9SBBif»T f — * * ©«9f*©ftttl»*tt-Srt" S IHS 
^ j=S i: , hu ISBf*^'— £ ^ ©^^(DfiHx— £ § 
tftiW¥tti#©i:» ftligB#iEijojf4a*iW|ii|fc^|.M|<o 2 «<D 

Mff^-^RST^frw^tHS^tbixL, fttfjatfrairT- 30 

-^^ffi^T, 15 2 fe<Dfflfi^'-£raT-ftffl*«*<D{itB 

f*tf S * ft 3 i: S WS^S***. T ^ 5 „ 
[0 0 10] ±ES 2 OBW«Mt 5ft»eSI*!( 3 

tc fc</ 1 T , iiij",E 2 ftcoiiiiif^-r— 3? >f </0W*O{ftiW £ 

[001 1] ,iii#vi4t;: I ia«wjeBij-ett. KMftftws 
f*^ttiSBAMix.?.ft, mitow&fo^mz.. mmicm so 



ltP)ftfe»»#St J: OjWStiftBftf-i" i:-?a6JS 
? ft tt»f tfn'«7*— ^ t ^tk^-r § p!i«fflBBK iot 

[0012] mm. 5 tc8B«oKWPt4, ii5itiS4^ia 

a«rfflV^Tgxf#^ft> SWaiBfiH-r— *t±">7 htw^ap** 
©tc J: K> m&t Wl& S ft T W# ^ ft S £> <D T o T , hU IB 

[0 0 1 3] »*3l6fc:aBK©aWTH:, *WcH\ II 

4 xtt»*3i 5 tc3M<DmmtoWi&miWH*. ?>ft 

fcBft-r- ^ i: S'^iBftT 1 '- ^ t ^ttlxt" •& MBit 
<fc D H«'r— ^ ft>©fM*tfttrtMftHi^afc <fc 0 tftlti * ft 

?>ft?>o jafien**j<oif4**riaMi:^MW2 

[0 0 1 4] I*S2lcEio»«l£J:ntf, w-kum 

zntzmmT- $ t mmmm^— * t ztmt § ®ft ® 

ate £ 0 Bff -r— £ * ©^{fcTbW^tb^Kfc «t "3 

S ft So ItiW^StC J: 0 EHt-r- ^ tfi(D^{*(7D^fctj^ 

tti^f^-^An^ai^ftSo WVi£¥St±, atiBB*iiiJOi , 44 
SlWli'lfc^MO 2 tfecoirtiflfe-r— *HU-?«ffi«lfr«3*ai» 
*tt$SL, ^^[SIUT-^ftif, 2ftWBft-r-^P H s 3 

tfm i: S'S: -S h # ti^m^frfc^Si^f*^ a-S ft 5 i: w 
[0 0 1 5] S»*JH3tcgE«<D«Mt«:«fcfttf, imWH\ 

m z ft s c sn 2 ©fij>i£¥a«: . w-aU^sk <t o «itu 

§ o 

[00 1 6] »^4fcE«o«Wfj:«fcntfs »*^ai 

S»ti, ¥Mk:Ktt6ft^»*#aK:«fe 0»»*ftfciB 
fife-r- * 4: ? i6M;^?ft/tM^i'!'i#x- * £ S 

iiiiifgisapKic iiw wisiS'.if^^^ft^m^sic <t 0 & 

tHTSo fit, ^oymm^m±Mfoh&mW*<nz 



(4) 
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<D ft ln-t± ,g ^ -T & S "P ^*'J 'Mi 2f tl § c 

[0017] 1 ;, , ]*x i :i5(c l ia ( iK^^0jj^j:ntf, w.'fWft 

[0 0 19] 

Si: LT^A^7 2«We.nn^o A^ 2kiC 
CD^ML, risBft'U l *^iM-r 20 

i^stift»*««*ilgf8o gib* new: A— 

[0 0 2 0] J*l£fH<DL CD^ 

OHgttBftO ^lfii»i:«^t 
§ 0 A^^zttHefc^-Tct^tcU^^/^d^L CD 

M-w-^nT^s (H7*#fisj o gift* nc 

si5 (0 4tc^T) «?ntt^ 0 WM^mmm 

rtttH u »*%#&ntf *<Dg;&f8»] trig 
-£3 m£5S5EfSSB 5 o«Bic»iS&o-a5*a tt^ 

§ o 

[0 0 2 1 ] i4tt, 5 cOiti«Wtf8JES*^ 

ftffiECU flt^SJSPSB) 6, *-tefflECU7, 

0, XfT'JV^tlft-fe^l K gflMl 2 
feJ;tfXt 0 -^ 1 3«^TV^o #— : J" try— ->a> 
yXfA(DM^ D a nT$§ ^-Ifffl ECU 7*3J;^ 

-*3«^ »&£SSii v t5fcffiji]snT^So ft 
u^m u/j h ^ - ^ *j uitr 5 c 1 1> 1? * 5 o 

[0022] ij^'y 2 C±B«JH RCU6 tCftftWKffific 

c c D*-p tcj: oiflt snftiif-?*B«ffl 50 



6 

ECU6tC$^rT£o #;*^2W:B{£fflECU6tcJ;9 
[0 0 2 3] irfijflWHE C U 6*Ct±. ^7h>t^>3>X 

h 1 4^utaasnsi:ife^ xfruy^n 

ft-b^l ltfA/D«»»l S^btSi^nt^ 
£ Q Sfc, iiifflECU6M, *-ttTfflECU7t 
S^MislK 1 2*<SagcSnT^5o A-tlifflECU 
7tCtt^*3**8N«S*U ^&Lffl[^§l 2tCtiXH° 

-a 1 3 3bW?ntt^ 0 

[0 0 2 4] 5/7b*^>3>^7f8tt, ^>7FU 
*£>tti*rtI*iS P SfccfcoTBflWHE C U6tt'>7 h# 

TP, R, N, D, 131, 25SJ «D6lg*^5o 

[0 0 2 5] F7X^7f9i^ WOF7 17 (0 

5tc*cr) oiwa*«MEp-r*t»o^ F7i 7<ommK 

iSmy • t7« F7fl§D S^M^o FTffi 
tJ-DS(D*> • t7W0{%^iIti: £ I;oT F7' 1 7 

[0 0 2 6] J?jS-fe>^ 1 *M^W*TSt>^ 

yy«tlft-fe>"9- 1 1 H\ »lte*H 8 (i7CSt) O 

+M His Plitf^f7y>W-^ (/n>F;W 1 
9 (H7^tkT) *$Kft57f7U>^>t7h (0 

[0 0 2 7] 7-fcfJIJE CU 7tt, ^-t^-y 3 
»JW*RlSfc«)<Dfe<DTa&So A— ^fcffflE C U 7«# 
«^»fJ»**rSo ***ff?S!T^. ij-ttlECU7 

a. BitfflE c u 6fr^9mx±izn%mm7 1 -$(Dm 

[0 0 2 8] Bftffl E C U 6 tt, **«ia«B (WT, 

cputi/^) 2 0, bam^u 2 1 ww^u cm 

7Ll£ R OM43<fctfR AM) 2 2^rf)iii^^o C P U 2 0 t± 
**rU 2 2^ffitfl*nfcyny5A-r—**cMr3l/^Ta 
fftSo &*3. I«fflECU6 (CPU 2 0) tCjcD. 

[0029] 2 ztctiflw^cfftii^ajsia^fi 1 3 

F7 1 7^WJLIfimcf glfi*l 
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[0 0 3 0] **fitefl^-et±. tftW£ftfc|5MM*AV » 

A\ »±»* ^ i: A^Sffi-T Sfr<D3 

fT-5o C P U 2 Ott, tfta»^HfliB3fflBO*E-&-e:0 

[0 0 3 1 ] ^^U 2 2tctt. m 1 ^7n-f-v-hT 

StlTV^So S^tC^U 2 2fc:«, y7h#^>3> 
SP^ Trj (Dt^ifC^e^ ^3cDMffi3 a*C»liB"eWL 

m s n s Bfc«* ftitiLft WW** MH ty» 

{*AHc£oT 2«Jg©«^J^^^So f¥L<&. 
[0 0 3 2] 05ti, »il3£l«fllStC*3ltS«**Sltftffi 

co «t n £ ^ -r i# w 0 r* * a 0 »ji ^*s® a <oyu^u 
t±x I'lD&'Ki AM^K (v = o) Lfcci^tfcSrrs^ 
c p u 2 oicx^mn^n^o *ei»*frc»»wi 30 

A^^*ftL?#SBf^BSW^. 2^rf^fijLT^ 
46 1 feCOSMff Image 0«:Btf#LT*5< o S*ffiff Ima 

— (JK*{fi[») rpj ^ — F^;l/ 

FT l 7AW**ns«^a*Bi«I»ageO*S»r-r 

cwsj to) ^it, efttiiiirtSkfc, M'i£i,v;oF7' i i 
[0033] m^Mmkte, *<on<oMmw5<rmmt ^ 
<o * ffii&fir a pts turn ^ ^ * * l t# § 0 

ff^lSajc, ^FU^-l 6^: rpj 

TNJ tc»ffrsoT% (f9v = 0) . y7h 

L/;^l 6^ rpj |/c^ FN J tcJftft^nfcBS^ffctf 
a«* K*PH ft Image 0 ^ Lti§ J: ^CbTl^o 
[0 0 3 4] Sfc. M'te).'iV<OF7' 1 7©IJHI¥JHJfWU M'K 

r mmcm o r # t # tc a — w»^*ats*r § 



§ $ Tic H* |IH#H o T ^ S i: # A** 5 «> & S o £ O 
£ * ^ libit *<OMi</ ^ft £\ JfcWfc:;*;* a^ftA** 0 , 
*>7hU/^1 TPj TNJ tcSfc^tlfefc 

$^^*Wft^»Mff Image 0 fc* S fcSHfttfJA^C 
TL£-3StlA*fe*o fct, FT 1 7 A^BflBH^tifci: 
#O»^PB«*^B«Lnage0 kLTMifract-pJC 

[0 0 3 5] *LT, v7h U/S— 1 6 TR j ^Atl 
%t (B**Jll) . RSMAt (i^if3 3-6 6^ 
'J 80 ftftCj^M&^U tt«»i« Image n (n=l, 
2, 3, ■«•) *SI5t5 0 it«SSi«Liagen*l«t!JS 
^^C. W£Mfc Image 0 k tb«iB«* Image n <h ^JrfrB^ 

±W*+^fM^*J ©5^if©fi-&^KKS-rSA^*ll 

Mf^n> $ivm^<^§^ (v^o) 5a5i^f¥ 

SffiPH^fi^LTfc, ;/7Ft°^>3>SPtf TRj 
^e^^3^Hjffi3 atc»^<DWH»A^WLtHSn*o 

[0036] ^cM^a^oi/^T^L<ra*r«o * 

2 teOHjft Image 0 , Image nSfSi» 
^tC^aOBfS^a^^So mx-li. Image n h Image 0 t 
COmcom^OM. (I Image n - Image 0 |) t <9 , 

^iB«tc«5S«ia*jfii'ro ct^a^ — 3o%m*amb 

ftR-r— ^Tlt TmageO ^ Image n 

[0 0 3 7] iif^tr^t § Ifflc 

i i A^tDfi^-fia s^bfbn/cS«i sowift 

8 t;,Wt^T— fift|iHi»±©^»iMJBiBS A^*i6a o 
«»i^^H^f5c H 6 ti, Zlfgaft Image 

tifcMiSi: LT*^bti§o H6 (a) ic^Tct^t, 

itjiot*©«n#ie (=o) thu -p«»3iiissp a 

(iii < H HWA^R < a S J: 5 »c f J o n't <*3tttf S*itt i: ft 
a 0 Sfc, 06 (b) ic^tXoic, /n>F;H 9?:W 
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t,\> &ib\ 0 6 (a) S««T?*SnS*aJ»fl: 
[0 0 3 8] «ffift*tt*<D*>rX* : PillL'ri»*«ii: 

ct 3 tC N $-Tz:ffiiB« Imaged fittS^SftilJtBSP A 

B fc:#»J U *"7n 7^B «fc,aAi£Snfcl»lffi*IHv\ 10 

l*^l':'l*fr«>T5fi£ 5 y V— JUft(DT—$ti\EW*l\fe-C 

m 6 tC^f ^&9&ifiifl!» P A (OffiWft<D*1>$i«0 20 

[0039] ^cm^vsmmnsMic^^rm i ^^n- 

[0 0 4 0] S^W^r*^ HffHIi:^[Hltc*3^SW 
^I^ffito c ct\ flutUOBf^iJ t -1 fc^I©«F*IJ r 

ftlftlCk.i-l (flU k=l, 2, -, Nt-1) . C 
k,i (fib. k=l, 2, NO tu 

co^r- k it&y Retell %Kr; i m*WAL^%ffi> J\ 

[0 0 4 1 ] lfXf77 c l 0T?«. «l£ti§&£:tt$$U 
N t - 1 = N t^fcSA^A^PJSrT So Nt-l^Ntfte fc£ 

mm^fofoiQi tPifet&o m2ic^?£5^ mm 
mm t tmmt-iT*<D&&&<Dmm&&T&mfoi*ic£ 

fcM^t^Tfijii£*T5 , PJAi-*afc«iar*o 
[0 0 4 2] Xf^y^ZO-Xf^y^SO^WJi, N 
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But* sfeftonaa^* o,^2 ow^afctHa^r 

Xf77°2 0m »H#$"Jt-l f t fcfcltSWWttKD* 
£«?:l:tit5 0 H**Jt fcB**Jt-iaDiihi«'r— ^rHI"^ 

X imm t-\(Dtp >L>mm c k . t - 1 1 , mm t<o $*>l>&m c 

k,tfc©!B^, (Ck.t-K Ck.O £$5<£, (C 
l,c-K Cl.t) , (Cl.t-K C2,t) , (Cl,t- 
U CNt.t) , (C2,t-K Cl.t) , (C2.t-K C2, 
c) . (C2,t-U CNt.t) . (CNt-l,t-K C 

1.0 . (CNl-1,l-U C2.0 , (CNt-l.t-K C 

N,t) <D£5K^T<Dm&&l£-2^TM'<Z>o *>CSffi«R 

. *^ffi«Sr (0, 0) fcLTitKT 

§ 0 ate. ^to^nsptw^ ca««) offin^ss 

[0 0 4 3] Xr77°3 0m -aSNc = 0T?*8 

frSMWKtSo B2 (a), (b) (c^fJ:3*c, 

(Ne = 0) o M2 (c) ICtt^XvIC Sihttl 

a t -pafb-a-r— a-rsot?. — s»n e *<*t?&< * 

§ (Ne^O) a JcoT. N e = 0 tfjSEiEt" *a*f. 

7 7°4 0T r»f«WJ fcW/iib, Ne^Otfetl 

[0 0 4 4] -J, Xr7 7° 1 0{C*3l^T. Nt-l = Nt 
tfeSi:*^ cnfcttTii: 3 fflii coffin© -5 ^ 4f*i 
tcKS"TS*^±»53rJ^*ftV\ Xr77°6 0, 7 0« 

Ste, Nt-i = Ntos&, s*r r^±w*co^j ^ 

[0 0 4 5] Xf77°6 Otlt S-^PSJ L -1, LfHfc:*5 

[0 0 4 6] Xf77°70T^ ^SNeMtN 
itClfL^ (Ne=Nt) ^S^WBft-So 13 3 (a) 

»Nc*^ttiaNt (=Nt-l) ^ A ^L<^So Sfc, H 
3 (b) , (c) {C^-r«t3tC, ^iEM%f*tf//ffiTi>^ 



(7) 
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oT, Ne=Nt (03 (a) (DM^ttN e = Nt= 2) 

tSo — S\ Ne-NtMub&^n^ (0 3 
(b) , (c) Xf7/9 otcitfro 

[0 0 4 7] "3^^ NM = NtOfct"e mWlM&& 
^£§^X-r^7°9 OtcilC^C £&C&S 0 Xr7 7° 

9o-xf7^i i o(omm^ Nt-i = Ntt* r^ft^ 

tCfll^T^o Xf7^9 0T*tt, -SSNc = 01?$5 
^^^Hit^o 0 3 (b) ic^TiStc. «rf*£>£ 

SiJt-lT-aftl-rsOT, — S»N e#*K4* (Ne = 
0) o Sfc. 0 3 (c) tcjjVT #itf&f*:*<ilfe 
■T5«-a\ »±»(*0*^««tt«f«lt-li:«F*J t T?£ 
{t-fr-T— St5©^ — &»N etf*T*ft<a« (Ne 
^0)o £oT, N e = 0 ifitiittftlS, Xf7 7°l 0 

1 ot r^ih^f*+^3»M*J fc*Jii£f So 20 

§ 0 $SSW±LT^7hU^-l 6^ TNJ 

rpj £An5ft££, v zicx-dX— fo<DW£Wi 

« Image 0^«»*tl*c HrVfik, Wfc3(£<0 FT 1 7# 
Rflffl^ftSliMc, S*PH« Image 0^S*ffStl5o Clft 
b coffiStc J: -3 r x * tcB*^*^^ U# 

S» tifil/^SBOjljfa^S^afS Image 0 £ LTHf*;* 

[0 0 4 9] eft^KOR»tt^tc«|LM*»ilJLi& 

^7hU^-l 6* Tpj Sfctt TNJ tcAft/c<h^O 
MfttfSViW* Image 0 fc^tl, **5#ft*¥Mfr5»9 

XPBHS Image 0 £ 2 ft £ 0 

[0050] ^cate#A^*ee*fftii'rsfc«)*c>'7 h 

UM— l 6^ r R j izAft&t, ft* 2te£-DTmtB 
2 ft tc ' t i Bfcf* ^ — * 3 CO litrj | fi i 3 a tcB£L£B£ 
ft^o 3<0|ifihfti3 atf^- f^f— i/ b > 

i 6^ r r j tcAft£ft£ HWWtftffiffl 
S£Wr£ft£o 

[0 0 5 1 ] T&fr^, PftfemrSk l S^ttKHftlmag 
en (n=l, 2, ••) jW«K*ft, *C0S*cS*PiB«I 
mage 0 httfttWISL Image n (n=l, 2, ■■■) <h^#:R 

M^i^ffl^^T^t-^ffifSMS^ff^ft. jaseiii 8 

?ft*i:, WSt&BiKiDa^ffSftSo 
[0 0 5 2] SfXf-yy (WT\ rs J <h!BT) 1 0 50 
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tfH2»c^-TJ:-5fcNt-i=jtNt©JJi'&ti:s 2 ofciffi*. 
H3tc^-TJ:3^Nt-l = Nt^a-tcS 6 0lcmts o 

[0 0 5 3] Nt,ltNt^)f^ |fS2 0t*L«S 
*tfcttl/T-a»NeS*»So S 3 0T*«— SK»N e 
-0t£§^Wt§o 02 (a) , (b) ^ 
t±5eNe = 0i!Di&tt r^fj]^j{*6D^j fcWSSft 
(S 4 0) , 0 2 (c ) iCTjkT e 0 <D±I^& 

r»±M*+©IMW*J i:M££ft£ (S5 0) o 

[0 0 5 4] ~^ Nt-l = NtOS^ $fS6 0W 
i^*«*itiKLr—aftN e s 7 or*tt-a 

IfcN c =NtT?&S^S*^*»JW'rSo 0 3 (a) ^ 

ftSo 03 (b) , (c) tCjjVTJfc-SlCN e ^Nt<Di§ 
&«U S9 0^Cjit? o S9 0m Ne = 0T*6WS 
fr*W»rt5o 0 3 (b) tC^t*J:5tCNe = 0Oiia- 

« mwi%nfo<Dfr\ tn^n (sioo) , H3 

(b) fcTfrt&oKN e*0<Dm&& r&±W»+&Ws 
¥M*J ^W^^ft^ (S110) o C5LT»*W±» 

[0 0 5 5] r»±»#0*J <D»^tt. Xb-^j 1 3 

7?. r^Oft^feOj wfti'&tt. xn-^ji 3^ero ! » : 
ft. *<Dje^»±*<*T?tt^t«i-> mimit-cmw* 

[0 0 5 6] mmtfWMV%&>2> £ (M&v * 0) . 

ai^Sfflpn.iD v° u if u \x m r -r s *^ • I wo^jy ^ t± 

^ ^ 3^imi¥il3 atcH^^^f^Jl^fe'^KLtB 
$fti> 0 fSiI«Ut^7hWN"l 6* TRj 

tc^JDi^^^ft^o 

[0 0 5 7] W±»5Ebfc«fc-5tc*||SS?gSBK:J:titf, 

(1) mmi-\, i<D2^(Dmm^-^rsxw.ii i AWt(Dm 

ttl»*lt'Rb (S 1 0) % ^tH^^^b<^^ (Nt-l 



(8) 
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2 Ca) ) tc r^iffij^feoj ©WAi-^fftiHHc-rsci: 

[0 0 5 8] ( 2 ) 0#£lJ t - 1 , t£) 2 tSc^Bf*^— £ fffl"? 

iftaw(t©*aia*it«Lrctt* (sio) , *WiR# 

(NM = Nt) fc^tt. £6^ifrffi«itR#L 

(S6 0) . -»»#«a»te«L^ 

(Ne = No r»±«5#o*j , -sssfewaa 

$U^L<%^ (Ne^Nt) r^|&M*&9J 

(sio) tt'iii^a-aawffiflwi! io 

(Nc=Ntt*5frSW (S 6 0, 7 0) £020 

[0059] (3) ^mmfotb*)} <omet. ^^mm 
co— (Ne = o) 6^s^*w»f-rsftia 
cs3o, S9 0) taot, mmwitvfri t m 

OlWlflJ 3 a ^D^i^P^M'^^tti^^mSS^^^^ 

C P U 2 0 <DffllOftS*6i(T**5o f$ih 
[0 0 6 0] (4) SiaS*ftilS^Sfcfe^5/7 FW^ 

rpj sfcti tnj tcxnfcfc^A^^ 

§ o 

[0061] (5) «nrt§ (g^rtS, S^fflia?) * 

[0062] -Mtmwut, ±,!e^KsnwF 
ow&tc r#ik*w*s>#j t ^m±^it^mh 50 
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cfcl/\> Xf77°l 5m NM=OSftttNt=0^ 

si-rsA^gft^wwrso H2 (a) K^fctsfc. b# 
*jc*«*j£«rr « c t a < *ffl»«ni^*?£»o«* 

[0 0 6 3] O WEHfiWBlKTtt. r»ik«rf*<D 

i: r^sbw** 0 j ^^2ffi:li r i^niyjiJt"i) 
J£»t?t> «fcl/\, $J^£0 9tc^r:7n— — h^S/IJ 
t*SSo 1"^^^ S 2 l 0fc*3l/>TNt.-l = Nt.ft^J« 

jz£>fc^&. S 2 2 0*^ r^iftftfe^j fcMStSo 

S/c, S 2 3 0T**fe-SRNe^ S 2 4 0lC*5^ 
TNe=NtT?*5i:WiShfefc#ttS 2 5 OT r» 
iH%{*O^J . Ne=Nt^ft^fcffl«;£nfcfc#l±S 

[0 0 6 4] O tftffi»#Offl«*lt« , r«fci6<0ffl« 
9J x. «a»flK<o«:K*P 6 & S § * wii^SST' t> J: 
— (1^) tcRB^^ Wl^.tf«mW«:*iEfflT? 

[0 0 6 5] O Kf'. l- f*!50*ftfflJ*ft«SMtt. ^filSBUJE 

[0066] o SttJr^s (ffi*or«) a r»ih»i*o 
[0067] o ffl»i75rffittJfiiVcaiii-e*ao rt/Sfflft'f 

r£tt L/c D , ^-^^ 3 c?)(itfilfil 3 a ^Wf^^ ^vf:*^ 
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